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This paper reports on results that complement those in an earlier 
paper by this writer which gives a constructive proof of the existence 
of an algorithm that, for each right-hand side a, produces a sequence 
which converges globally and superlinearly to a solution x off(x) = 
a whenever f is a C 1 -diffeomorphism (i.e., is a continuously-differen- 
tiable invertible map with continuously-differentiable inverse) of a 
Banach space B onto itself and either B = R n or f satisfies certain 
other conditions that are often met in applications. Here we consider 
the case in which f is Lipschitz on each bounded subset of B. We 
give results which, while along the lines of those obtained earlier, 
concern a fundamentally different Newton-direction algorithm which 
does not appear to have been introduced previously, and which has 
the advantage that its implementation does not require the use of 
certain search procedures. 

I. INTRODUCTION 

Let f be a function from U into B, where B is a Banach space with 
norm | • |, and U is a nonempty open subset of B. We say that f is 
differentiate on a set S C U if /has a Frechet derivative f'(s) at each 
point s of S.f (If, for example, B = R" with the usual Euclidean norm, 
then / is differentiate on U if it is continuously differentiable on U in 
the usual sense.) By / a C l -diffeomorphism, we mean that / is a 
homeomorphism of U onto B, and f and (f~ l Y exist and are continuous 
on U and B, respectively. (We emphasize that here continuity refers to 
the dependence of the derivatives on the points at which they are 



* This paper was presented at the Fourteenth Asilomar Conference on Circuits, 
Systems, and Computers (Pacific Grove, California, November 17-19, 1980). 

f In other words, f is differentiable on S C U if for each s G. S, there is a bounded 
linear map f'(s):B -» B such that f{s + h) = f{s) + f'(s)h + o(\h f) as \h | -+ 0. 
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evaluated, not to their boundedness as operators, which is assured by 
definition*) C'-diffeomorphisms frequently arise in applications. 

The purpose of this paper is to report on results that complement 
those in Ref. 1 where a constructive proof is given of the existence of 
a Newton-direction algorithm that, for each a G B, generates a se- 
quence in U which converges globally and superlinearly to a solution 
x off(x) = a whenever /"is a C^diffeomorphism of U onto B and either 
B = R" or /satisfies certain other conditions that are frequently met 
in applications. (For the case of an important class of monotone 
diffeomorphisms fin a Hilbert space H, the "other conditions" reduce 
to simply the requirement that /' be uniformly continuous on closed 
bounded subsets of H. A specific example in which H is infinite 
dimensional is given in Ref. 1.) 

The algorithm described in Ref. 1 typically involves the recursive 
determination of positive scalars y , Yl • • • (which determine the 
successive steplengths) such that a certain ratio R k (y*) (which depends 
on the kth iterate x k ) lies between prescribed bounds for all k = 0, 1, 
2, • • • . While it is proved that the y* can be chosen as required, and 
that y k - 1 for all sufficiently large k, the actual determination of the 
y* in a specific case would ordinarily require the use of a one-dimen- 
sional search procedure for a finite (and possibly large) number of 
values of k. 

In this paper we address the case in which U = B and f is Lipschitz 
on bounded subsets of B (i.e., is such that for each bounded subset S 
of B there is a constant A such that | f'(u) - f'(v) | < A | u - v | for all 
u and v in S). We give results which, while along the lines of those in 
Ref. 1, concern a fundamentally different Newton-direction algorithm 
that does not appear to have been introduced earlier, and which does 
not require the use of search procedures to solve subproblems of the 
type outlined above. 

Our results are presented in Section II. As a consequence of the 
Lipschitz hypothesis, proofs are comparatively simple and we are able 
to establish quadratic (rather than superlinear) convergence. (Recall 
that a sequence x\ x 2 , • • • inB converges quadratically to an element 
x of B if the sequence converges to x and there is a constant c such 
that \x k+i - x|< c\x k - x\ 2 for all A. T ) 

General relationships between diffeomorphisms and computation of 
the type described in Ref. 1 and in Section II do not appear to have 



* And of course, this continuity is with respect to the usual induced norm of a 
bounded linear map of B into B. 

f Quadratic convergence results follow easily from those in Ref. 1 under the Lipschitz 
hypothesis used here. (In this connection, see the last part of the proof of Lemma 1 in 
Section II.) 
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been reported on earlier by other writers. On the other hand, as in 
Ref. 1 our approach involves the minimization of a functional, and 
therefore in a general sense there is a vast related literature. [See, for 
example, Ref. 2 and note (p. 190) that the least-squares Newton- 
direction methods described there require, in particular, the existence 
of second derivatives of f (our notation).] Additional background 
material can be found in Ref. 1. 

II. PROCESSES No and N, 

Throughout this section we use the terms Lipschitz and converges 
quadratically in the way indicated in Section I, we denote the usual 
induced norm of a linear map A of B into B by \A |, and we take U = 
B. 

With /differentiable on B, but not necessarily a C^diffeomorphism, 
and with x a and a any two elements of B, consider the following 
process, in which s k denotes | f{x k ) - a | whenever x k E B is defined. 

Process Ni : Choose p E [V6, 1) and A > 0. Do the following for k = 
0, 1, • • • 

lif{x k ) = a,setx k+1 = x k . 

If f(x k ) t^ a, determine (jn£B such that 



/•'(**)</» = a -/"(**)• Then 
1. Let y k = (As*) -1 if 



s k > 2pA -1 



= 1 if s*<2pA -1 . 

2. Lety* +1 = x k + y k <t>k. 

3. Set x k+1 = y k+} if either s* > 2pA-' and \f(y k+l ) - a|< [1 - 
(2As A ) _1 ]s A , or s k < 2pA _1 and | f(y k+i ) - a \ < Vfe As!. If neither pair of 
conditions is met, replace A by 2A in Step 1 and the sentence preceding 
this sentence, and return to Step 1. 

Our main result is the following. 

Theorem 1: Suppose that f is a C x -diffeomorphism ofB onto B. Let 
f be Lipschitz on bounded subsets of B, and let \ (f~ 1 )' | be bounded 
on bounded subsets of B. Then for each a and each x°, Process N\ 
can be carried out, and x\ x 2 , ••■ converges quadratically to the 
unique solution xoff{x) = a. 

2. 1 Proof of Theorem 1 

Let a and x° be given. 

We first prove two lemmas which concern cases in which /"need not 
be a C -diffeomorphism. Let L = {v G B:\f(v) - a\ < \f(x°) - a\), 
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and let L denote {w + a f'(w)~\a - fjw)]:w G L, a G [0, 1]} when 
f'(')~ l exists on L. (Assuming that L is defined, notice that it is 
bounded if L is bounded and | /'( • ) _1 1 is bounded on L. This observa- 
tion is used later in the proof of Theorem 1 and in connection with 
Lemmas 1 and 2, below.) With r\ a positive constant, consider the 
following process. 

Process N : Choose p G [%, 1). Do the following for k — 0, 1, ■ • •. 

If /"(**) = a, set x k+l = x k . 

If f(x h ) ¥* a, determine fa G B such that 

f'(x k )fa = a - /(**)■ Then let 

Y* = (t?s*) _1 if Sk > 2ptj~ i 
= 1 if s k < 2pif *. 

Setx* +1 «** + wfo. 



Lemma 1: Assume that L is bounded and thatf'i- ) andf'( • ) _1 exist 
on L with | f'(w)~ l | < Kfor wEL and some constant K. Assume also 
that /'(•) exists and is Lipschitz, with Lipschitz constant A, on L. 
Then for tj > AK 2 , 

(a) Process N can be carried out, and for each k such that Sk¥=0 
we have 

s k+i < [1 - (2tjs a )- 1 ]s* if s*>2pif\ (1) 

s k+i < V6 tjsI < ps k if s* < 2prf '. (2) 

(6) s k -* as k — » oo. 

(c) //" f/iere is an x G B such that f(x) = a and x k — » x as k — > oo, 
#ien {x*} converges quadratically. 

2.1.1 Proof of Lemma 1 

We will use the following proposition. 

Proposition 1: Suppose that the hypotheses of Lemma 1 are met. If 
x k G L, and y G [0, 1], and fa denotes f'(x k )~\a - f(x k )], then, for tj 
> AK 2 , we have \f(x k + yfa) - a\ < (1 - y) \f(x k ) -a\ + ^y 2 \f(x k ) 
-a | 2 . 

Proof: We have 

|/V + yfa) - a\ = |/V) - a + f'(x k )yfa + 8\ 
in which 

5 = f(x k + yfa) - f(x k ) - f'(x k )yfa. 
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Thus 



and 



\f(x k + y«fo) - a\ < (1 - y) |/(**) - a| + |5|, 



Jo 

Since | S | < x hhy 2 K 2 \ f(x k ) - a | 2 , we have proved the proposition.* 

Assume now that -q>KK 2 , and that the hypotheses of Lemma 1 are 
met. 

Let k be such that either k = 0, or Process No can be used to generate 
x 1 , • • •, x* with x 7 £ L fory = 1, 2, • • •, k. Suppose that s k ^ 0. Since 
x* G L, fa can be determined. Since p £ [Ms, 1), when s* > 2prj _I we 
have (tjsa)" 1 < 1. Thus, by the proposition, (1) holds. On the other 
hand, obviously Vfcrjs* < p when s k < 2piT\ and thus, by the proposition, 
(2) is met. This shows that x k+1 can be determined, that it satisfies (1) 
and (2), and that x* +1 e L, which proves Part (a). 

Part (b) is a direct consequence of Part (a), because, by Part (a), if 
s k does not approach zero as k -> co we must have s* > 2prj _1 for all k 
in which case [1 - (2tjs ) -1 ] e (0, 1) and s k < [1 - (2tjs ) _1 ]*s for k > 
1, which is a contradiction. 

Assume now that B contains an x such that f(x) = a and x* -> x as 
k -* oo.f Since x k E. L for all k, and L is closed, x £ L. Let J denote 
/"'(x). Since J is an invertible bounded linear map of B into itself, there 
are positive constants /Si and /? 2 such that fix \ u \ < | Ju \ < ^ 2 1 u \ for w 
£ B. For each ^, we have 

\f(x") -a\ = \ fix) -a + J(x k - x) + 8 k \ 
in which | 5* | (|x* - x|) _1 -* as k -+ co. 
Notice that for some m, 

|J(x*-x)|>2|8*| for k > m. 
Thus for ^ > m, 

|/(x*) - a | > | J(x" - x) | - | S k | > Vi | J(x* - x) | > l ^, | x* - x |, 

and, on the other hand, 

|/(x*)-a|<| e /(x*-x)| + |5.|< 3 /2|J(x A -x)|<y2 i 8 2 |x*-x|. 

We have s k +i £ '^tjsa for ^ > M for some M>m. 



* With regard to the origin of the formula for y* in Process iV 0) notice that the right 
side of the main inequality of the proposition is minimized with respect to y at y = 

t The existence of such an x follows from our hypotheses, but this fact is not needed 
for our purposes. 
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Therefore, 

\x k+] -x\ <%T)fc x ($l\x k -x\\ k>M, 

which completes the proof of the lemma. *f 

Lemma 2: Suppose that L is bounded, and that /"'(■) and /'(-J -1 
exist on L with \ f'( ■ ) _1 1 bounded on L. Suppose also that /"'(•) exists 
and is Lipschitz on L. Then Process N\ can be carried out, we have 
Sk — * as k — > oo, and if there is an x £fl such that f(x) = a and x k 
— » x as k —*■ oo, then x 1 , x 2 , • • • converges quadratically. 

2. 1.2 Proof of Lemma 2 

Consider Process Ni. By Lemma 1, there is a constant A that 
depends only on f, a, and x° such that if A in Step 1 and the first 
sentence of Step 3 satisfies A > Ao, and if either k = and So ^ 0, or k 

> and Process N\ can be used to determine x k with s* ^ and s* < 
So, then Step 3 can be carried out on the first pass. Notice that 
whenever x k+1 is set equal to y k+i in Step 3, we have sa+i < s*. 

Since for any A > there is a nonnegative integer p such that 2 P X 

> Ao, it follows that Process Ni can be carried out, and that for some 
nonnegative integers q and r, we have s h+1 < ai,s k for k > q, where a* 
= [1 - (2 r+, As 7 )-'] when s k > 2p(2 r A) -1 and a k = p otherwise. Since 
Ok < 1 for k > q, it is clear that s* — » as ^ — > oo, and therefore that s/,+i 
< '/2 2 r As* fork>M for some M. Thus, by the proof of Part (c) of 
Lemma 1, our proof of Lemma 2 is complete. 

Now let the hypotheses of Theorem 1 be met. The proof of Theorem 
3 of Ref. 1 shows that L is bounded, that /'(•)' exists on B, and that 
I /'(•)"' | is bounded on L. Since /"is a homeomorphism of B onto B, Sk 
— > as k — * oo implies that x k — > x as k — » oo, where x satisfies f(x) = 
a. By Lemma 2, this completes the proof of Theorem 1. 

2.2 Monotone diffeomorphisms in Hilbert space 

Let ^:[0, oo ) —* [0, oo) be continuous, strictly increasing, and such 
that i//(0) = 0, i//(a) —> oo as a — » oo, and a~V(«) ^ c for a £ (0, a ) for 
some positive constants c and a. Notice that, for example, \p(a) = a 
meets these conditions. 



* The fact that {x*} converges quadratically follows from a direct extension of a 
known result (see Ref. 3, p. 312) since either s* = for some k, or there is an M such that 
y* = 1 for k > M. The short proof given above is included for the sake of completeness. 

t D. J. Rose has informed this writer that in recent independent joint work with R. 
Bank, 4 done subsequent to the appearance of preprints of Ref. 1, a corresponding result, 
as well as a result corresponding to Theorem 1, was obtained for a process in which y* 
= (1 + r\kSk)~\ where the Vk satisfy certain inequalities. They study a case in which an 
approximation M k to f'(x k ) can be used in place of f'(x k ). Also, earlier related work 
along different lines concerning uniformly monotone gradient maps f:R"—*R" was done 
by Bank and Rose. s 
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Theorem 2: Let f map a real Hilbert space H, with inner product 
(., •), into itself such that (f(u) - f(v), u- v) >\u- v\^(\u - v\) 
for all u, v G H. Assume that f exists and is Lipschitz on bounded 
subsets of H. Then f is a C 1 -diffeomorphism of H onto H, and the 
conclusion of Theorem 1 holds. 

Using Theorem 1, a proof of Theorem 2 can be obtained by trivially 
modifying the proof of Theorem 4 of Ref. 1. 

2.3 B= R n 

The following complete result is a direct corollary of Theorem 1 (see 
the proof of Theorem 5 of Ref 1). 

Theorem 3: Let B = R", and let f be Lipschitz and continuously 
differentiable on bounded subsets ofR". Then f is a C 1 -diffeomorph- 
ism ofR n onto itself if and only if 

(i) Process AT, can be carried out for each a and each x°. 

(ii) For each a, the sequence produced by Process Ni converges 
quadratically to a solution x off(x) = a, and x does not depend on x°. 

2.4 Comments 

As in Ref 1, our primary purpose is to focus attention on general 
relationships between diffeomorphisms and computation. Clearly, no 
attempt is made to optimize the performance of all aspects of the type 
of algorithm described. However, there are some basically self-evident 
modifications that are sometimes useful. For example, the total num- 
ber of iterations required in a specific case can sometimes be reduced 
significantly by repeatedly, or occasionally, stopping the algorithm 
after a number of steps and resetting the initial value of A in Process 
Ni to a smaller number. (It is not difficult to give rules of thumb 
concerning when to stop the algorithm and by how much to reduce A, 
but we have not tried to prove theorems that bear on these matters.) 
Of course, bounds on the location of the solution and estimates of K 
and A, which are available in some problems, can be used in an obvious 
way. Similarly, if for example B = R", a globally convergent steepest- 
descent process (see Ref. 6) * might be used initially to obtain a better 
approximation to the solution before the Newton-direction algorithm 
is used. (In fact, a well known and often useful strategy is to combine 
steepest descent and pure Newton iterations in this way.f) 



* We take this opportunity to correct a typographical error in Ref. 6. On page 1004, 
left column, line 2, [2] should be replaced with [21]. 

t This paragraph was motivated by a helpful observation by D. J. Rose to the effect 
that, as the algorithm stands, there are cases in which many iterations are required. 
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